Correcting Decalibration of Stereo Cameras in Self-Driving Vehicles

We address the problem of optical decalibration in mobile stereo camera setups, especially in context of autonomous vehicles. In real world conditions, an optical system is subject to various sources of anticipated and unanticipated mechanical stress (vibration, rough handling, collisions). Mechanical stress changes the geometry between the cameras that make up the stereo pair, and as a consequence, the pre-calculated epipolar geometry is no longer valid. Our method is based on optimization of camera geometry parameters and plugs directly into the output of the stereo matching algorithm. Therefore, it is able to recover calibration parameters on image pairs obtained from a decalibrated stereo system with minimal use of additional computing resources. The number of successfully recovered depth pixels is used as an objective function, which we aim to maximize. Our simulation confirms that the method can run constantly in parallel to stereo estimation and thus help keep the system calibrated in real time. Results confirm that the method is able to recalibrate all the parameters except for the baseline distance, which scales the absolute depth readings. However, that scaling factor could be uniquely determined using any kind of absolute range finding methods (e.g. a single beam time-of-flight sensor).Comment: 8 pages, 9 figure

A time-motion analysis of turns performed by highly ranked Viennese waltz dancers

Twenty-four dance couples performing at the 2011 IDSF (International DanceSport Federation) International Slovenia Open were divided into two groups: the first twelve placed couples (top ranked) and the last twelve placed couples (lower ranked). Video recordings were processed automatically using computer vision tracking algorithms under operator supervision to calculate movement parameters. Time and speed of movement were analysed during single natural (right) and reverse (left) turns performed during the Viennese waltz. Both top and lower ranked dancers tended to perform similar proportionate frequencies of reverse (≈ 35%) and natural (≈ 65%) turns. Analysis of reverse turns showed that the top ranked dancers performed less turns on a curved trajectory (16%) than the lower ranked dancers (33%). The top ranked couples performed all turns at similar speeds (F = 1.31, df = 3, p = 0.27; mean = 2.09m/s) all of which were significantly quicker than the lower ranked couples (mean = 1.94m/s), the greatest differences found for reverse turns (12.43% faster for curved trajectories, 8.42% for straight trajectories). This suggests that the ability to maintain a high speed in the more difficult turns, particularly the reverse turns on a curved trajectory, results in the overall dance appearing more fluent as the speed of movement does not fluctuate as much. This aspect of performance needs to be improved by lower ranked dancers if they wish to improve rating of their performance. Future research should determine which factors relate to the speed of turns

Using a situation awareness approach to identify differences in the performance profiles of the world’s top two squash players and their opponents

Purpose The pressure exerted on a squash player is a consequence of the quality of a shot coupled with the ability of the player to return the ball, namely, the coupling of the two players’ situation awareness (SA) abilities. SA refers to an awareness of all relevant sources of information, the ability to synthesize this information using domain knowledge and the ability to physically respond to a situation. Methods Matches involving the two best players in the world (n = 9) at the 2011 Rowe British Grand Prix, held in Manchester, United Kingdom were recorded and processed using Tracker software. Shot type, ball location, players’ positions on court and movement parameters between the time an opponent played a shot prior to the player’s shot to the time of the opponent’s following shot were captured 25 times per second. All shots (excluding serves and rally ending shots) produced five main SA clusters, similar to those presented by Murray et al. (2018), except a greater proportion of shots were categorized in the greater pressure clusters and less in the lower pressure ones. Results Individual matches were presented using cluster performance profile infographics which demonstrated how individual player’s performance profiles differed between matches. Conclusion It is suggested that it is the coupling, of the two player’s behaviors, that makes the examination of tactics so challenging. This inherently means that performance profiles vary in subtle ways, making consistent profiles that are independent of the opponent very unlikely for elite players. This approach should be further modified to determine within match changes in performance

1st Workshop on Maritime Computer Vision (MaCVi) 2023: Challenge Results

The 1$^{\text{st}}$ Workshop on Maritime Computer Vision (MaCVi) 2023 focused on maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicle (USV), and organized several subchallenges in this domain: (i) UAV-based Maritime Object Detection, (ii) UAV-based Maritime Object Tracking, (iii) USV-based Maritime Obstacle Segmentation and (iv) USV-based Maritime Obstacle Detection. The subchallenges were based on the SeaDronesSee and MODS benchmarks. This report summarizes the main findings of the individual subchallenges and introduces a new benchmark, called SeaDronesSee Object Detection v2, which extends the previous benchmark by including more classes and footage. We provide statistical and qualitative analyses, and assess trends in the best-performing methodologies of over 130 submissions. The methods are summarized in the appendix. The datasets, evaluation code and the leaderboard are publicly available at https://seadronessee.cs.uni-tuebingen.de/macvi.Comment: MaCVi 2023 was part of WACV 2023. This report (38 pages) discusses the competition as part of MaCV

Joint Calibration of a Multimodal Sensor System for Autonomous Vehicles

Multimodal sensor systems require precise calibration if they are to be used in the field. Due to the difficulty of obtaining the corresponding features from different modalities, the calibration of such systems is an open problem. We present a systematic approach for calibrating a set of cameras with different modalities (RGB, thermal, polarization, and dual-spectrum near infrared) with regard to a LiDAR sensor using a planar calibration target. Firstly, a method for calibrating a single camera with regard to the LiDAR sensor is proposed. The method is usable with any modality, as long as the calibration pattern is detected. A methodology for establishing a parallax-aware pixel mapping between different camera modalities is then presented. Such a mapping can then be used to transfer annotations, features, and results between highly differing camera modalities to facilitate feature extraction and deep detection and segmentation methods

Human-centered deep compositional model for handling occlusions

Despite their powerful discriminative abilities, Convolutional Neural Networks (CNNs) lack the properties of generative models. This leads to a decreased performance in environments where objects are poorly visible. Solving such a problem by adding more training samples can quickly lead to a combinatorial explosion, therefore the underlying architecture has to be changed instead. This work proposes a Human-Centered Deep Compositional model (HCDC) that combines low-level visual discrimination of a CNN and the high-level reasoning of a Hierarchical Compositional model (HCM). Defined as a transparent model, it can be optimized to real-world environments by adding compactly encoded domain knowledge from human studies and physical laws. The new FridgeNetv2 dataset and a mixture of publicly available datasets are used as a benchmark. The experimental results show the proposed model is explainable, has higher discriminative and generative power, and better handles the occlusion than the current state-of-the-art Mask-RCNN in instance segmentation tasks